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Executive summary 



Introduction 

The Mark Friedman Outcomes Based Accountability 
(OBA) approach starts with a focus on outcomes and 
provides a framework for planning and performance 
managing services. The OBA model has been used in 
the USA and several countries worldwide as a way of 
structuring planning to improve outcomes for whole 
populations and for improving services. The OBA 
approach focuses on outcomes that are desired and 
monitoring and evidencing progress towards those 
desired outcomes. Key features of OBA include: 

• population accountability, which is about improving 
outcomes for a particular population within a 
defined geographical area 

• performance accountability, which is about the 
performance of a service and improving outcomes for 
a defined group of service users. 

Another key feature of OBA is the use of performance 
management categories which distinguish between 
'Flow much did we do?', 'Flow well did we do it?' and, 
the most important category, 'Is anyone better off?'. 

Local Government Improvement and Development (LG 
Improvement and Development, formerly the 
Improvement and Development Agency (IDeA)) has 
invested significant resources over a number of years 
to support local authorities (LAs) in implementing the 
OBA approach. LG Improvement and Development 
commissioned a survey assessing the initial impact of 
OBA on improving outcomes for children and young 
people (undertaken by Local Government Association 
Analysis and Research (LGAAR)).' This study, 
commissioned by LGAAR, on behalf of LG Improvement 
and Development, is the second element of this 
research. 



Aims and objectives 

The National Foundation for Educational Research 
(NFER) was asked to identify the impact of the OBA 
approach on LA practice in Children's Services. The 
specific objectives were to identify: 

• a sample of LA Children's Services authorities at 
different stages of implementing the OBA process 

• any changes and improvements to outcomes as a 
result of using OBA 

• examples of success and the factors that contributed 
towards this 

• if implementation of OBA had not led to an 
improvement in outcomes to further explore why this 
was the case 

• challenges and barriers and how these were 
overcome. 

Methodology 

NEER conducted one-to-one interviews with 18 
strategic and operational stakeholders and collected 
supporting background documentation from nine LAs 
(one from each Government Office region). Each of the 
nine LAs formed a 'case-study' for this research. 

Summary of findings 

The research found evidence to suggest that, at a 
comparatively early stage in its use, OBA was 

contributing to improved outcomes for children 
and young people in a few instances. In the 

main, however, at this stage the use of OBA was 

having an impact on working practices and 

indirectly on relationships between Children's Services 
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and partners in the Children's Trust as the Turning the 
Curve workshops provide an opportunity for partners to 
come together with a common focus. Staff who had 
been using the OBA approach believed that, over time, 
this would lead to improved outcomes for children and 
young people. 

Although the interviewees had yet to identify a clear 
relationship between the use of OBA and improved 
outcomes for children and young people, in general, 
they were looking at making greater use of OBA 
across Children's Services. This was primarily 
because they had found the process of undertaking a 
Turning the Curve workshop a helpful methodology 
for structuring thinking and moving towards 
becoming more outcomes-focused. Some 
interviewees also found the outcome grids and report 
cards useful, although these were less widely used. 

OBA was often regarded as one of a range of 
management tools available, but one which did not, 
however, deal with cost effectiveness. 

As they were adopting the OBA methodology in a 
Children's Services and Children's Trust setting, the 
interviewees were learning and, through their fuller 
understanding, adapting the approach to enable 
them to make best use of it. This included selecting 
the elements of the approach that assisted them in 
identifying outcomes and improving performance. 

The research suggests that the process of implementing 
OBA benefited from the support of Directors of 
Children's Services and equivalent Tier 1 (director- 
level) officers from across the Children's Trust partner 
organisations. Where the OBA approach was being 
applied to the review and development of the Children 
and Young People's Plan (CYPP) and to the process of 
commissioning services and contract monitoring there 
was more likelihood of OBA becoming embedded in 
Children's Services practices. 

The evidence from the interviews suggests that while 
OBA can appear initially to be a simple approach for 
focusing on outcomes, fuller engagement with the 
approach showed that it is complex. 

Consequently, it is evident that in adopting OBA, 
Children's Services authorities needed to allow time 
to understand fully the concepts and 
terminology and to ensure these are mutually 
understood. In all nine case-study areas training had 



been undertaken in OBA and it appeared that this 
had been valuable in assisting in the understanding of 
the process. 

Issues for consideration 

Based on the evidence collected, a number of issues 
emerged at local level which are worthy of 
consideration in supporting Children's Services 
authorities in implementing OBA. Although it is too 
early to assess the full impact of OBA at this stage, 
should Children's Services authorities be encouraged to 
implement OBA, the following issues are worthy of 
consideration at local level. 

• Gain commitment at high strategic level and engage 
with key partners. 

• Ensure those leading the implementation are trained 
and fully understand the approach so they are able 
to support others. 

• Ensure there is clarity of the aims and reasons for 
implementing OBA and whether it is being used to 
focus on population outcomes or specific services. 

• Involve as many partners as possible in Turning the 
Curve workshops to support shared ownership of the 
approach. 

• Ensure there is a common understanding of the OBA 
language: for example, what an outcome is; the 
difference between 'effort' and 'effect'. 

• Ensure those leading the Turning the Curve 
workshops are fully prepared and aware of what 
data is available. 

• Keep practising using the outcomes grids iot 
performance accountability - they are dynamic 
documents which can be improved. 

• Dedicate time for relevant personnel who are going 
to be using OBA to engage with it and 'champion' it 
across the Children's Trust. 

• Begin by introducing OBA in one or two specific 
service areas or focus on particular population 
outcomes. 
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• Build OBA into existing systems/strategies to ensure 
it becomes embedded. 

• Embed OBA in performance management by using 
the OBA outcomes grids to focus monitoring on 
outcomes and how these will be measured. 

• Build OBA into a key officer's role within the LA or 
Children's Trust - someone who can act as a 
champion and maintain the wider commitment to 
the approach. 



• Use examples of success of using OBA to help gain 
more universal commitment. 

Note 

1 See http://www.idea.gov.uk/idk/aio/16109926 (EGA 
and IDeA, 2009) for a copy of the report on the 
survey findings. 
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1 Introduction 



1.1 Background to OBA 

With the introduction of Every Child Matters: Change for 
Children (HM Government, 2004) came a greater focus 
on thinking about outcomes for children and families. 

The Mark Friedman Outcomes Based Accountability 
(OBA) approach (Friedman, 2000) (or results-based 
accountability as it is referred in the USA and other 
countries) starts with a focus on outcomes and 
provides a framework for planning and performance 
managing services. 

The OBA model has been used in the USA and several 
countries worldwide as a way of structuring planning 
to improve outcomes for whole populations and for 
improving services. The structure provided by OBA 
provides individuals, teams and groups of stakeholders 
with a framework for thinking about, and planning 
action, to improve outcomes. The model emphasises 
the importance of using a shared and common 
language across partner agencies. 

The OBA approach focuses on outcomes that are 
desired and monitoring and evidencing progress 
towards those desired outcomes. Key features of OBA 
include: 

• population accountability, which is about improving 
outcomes for a particular population within a 
defined geographical area 

• performance accountability, which is about the 
performance of a service and improving outcomes 
for a defined group of service users. 

Using the OBA approach entails the use of a Turning 
the Gvw workshop. This involves identifying the 
desired outcome, projecting what would happen over 



time if nothing changed, and then planning changes 
which will lead to improved outcomes or 'turned 
curves' that move away from the initial projection. A 
second key feature of OBA is the use of performance 
management categories which distinguish between 
'Flow much did we do?', 'Flow well did we do it?' and, 
the most important category, 'Is anyone better off?' 
(see Box 1.1). 

Box 1.1 Use of performance management 
categories in OBA 

How much did we do? How well did we do it? 

(the quantity of (the quality of 

the effort) the effort) 



Is anyone better off? 

(the quantity and quality of the effect) 



Local Government Improvement and Development (LG 
Improvement and Development, formerly the 
Improvement and Development Agency (IDeA)) has 
invested significant resources over a number of years 
to support LAs in implementing the OBA approach. LG 
Improvement and Development commissioned a survey 
assessing the initial impact of OBA on improving 
outcomes for children and young people (undertaken 
by LGAAR).^This study, commissioned by LGAAR, on 
behalf of LG Improvement and Development, is the 
second element of this research. 

Note 

2 See http://www.idea.gov.uk/idk/aio/16109926 (LGA 
and IDeA, 2009) for a copy of the report on the 
survey findings. 
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2 Research methods 



2.1 Aims and objectives 

NFER was asked to identify the impact of the OBA 
approach on LA practice in Children's Services. The 
specific objectives were to identify: 

• a sample of LA Children's Services authorities at 
different stages of implementing the OBA process 

• any changes and improvements to outcomes as a 
result of using OBA 

• examples of success and the factors that contributed 
towards this 

• if implementation of OBA had not led to an 
improvement in outcomes to explore further why this 
was the case 

• challenges and barriers and how these were 
overcome. 

2.2 Methodology 

NFER conducted one-to-one interviews with 18 
strategic and operational stakeholders^ and collected 
supporting background documentation from nine LAs 
(one from each Government Office region). The nine 
LAs were selected from those who responded to the LG 
Improvement and Development survey assessing the 
initial impact of OBA and indicating that they would be 
willing to participate in further research in this area. 

The LAs were selected to ensure a range in terms of 
council type and service areas where OBA was being 
applied. The focus and content of the interview 
schedule was discussed and agreed with LGAAR. 

All nine LAs provided supporting documentation which 
included notes and actions from OBA Turning the Curve 
workshops, CYPP priorities, OBA project reports and 



evaluations, and data giving evidence of improved 
outcomes for children and young people. 

Of the 18 interviewees who participated in this 
research study, 13 were in commissioning, planning or 
performance roles, four held service specific roles and 
one a specific change management role. 

Each of the nine LAs formed a 'case study' for this 
research. Data from each case study was also analysed 
and a separate report for each is available. Each case 
study focused on how OBA had been used in that LA 
or by the Children's Trust. Information is included on 
implementation and impact, including highlighting 
learning and how to overcome OBA barriers and 
challenges. 

This report is based on a thematic analysis of all of the 
interview data using Max ODA - a software package 
to support complex and large-scale analyses of 
qualitative data - to provide an overview of the 
process and impact of using OBA. 

2.3 Structure of report 

This report begins by summarising how OBA was being 
used in the nine Children's Services authorities. It then 
provides an analysis of the impact of using OBA and 
the perceived challenges and barriers associated with 
this. The report concludes with a summary of the 
findings emerging from the evidence. 

Note 

3 This included LA officers from Assistant Director at 
Tier 2 level, heads of service at Tier 3 level and 
programme managers at Tier 4 level. 
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3 How was OBA being used? 



This section reports on OBA as a process and covers 
why Children's Services authorities were using OBA, 
who was involved, what it was being used for, how 
long it had been used for and how it had been 
introduced. The impact of using OBA is discussed in 
Section 4. 

3.1 Why use OBA? 

In all nine areas the main reason why Children's 
Services authorities had implemented OBA was to help 
them become more outcomes-focused. OBA was seen 
as a tool to assist with achieving this, as the following 
three guotations demonstrate: 

The appeal of focusing on outcomes was key to it. 

(Senior Joint Commissioning Manager) 

/ think because it fitted very well with our strategy. Our 
strategy Is to be outcomes-focused and this was a tool 
which helps you to be outcomes-focused. 

(Divisional Manager for Joint Commissioning) 

It hit us at a time that we needed to be more outcomes- 
focused and we had a real need to make better use of our 
performance information. 

(Performance and Improvement Manager) 

Overall, interviewees felt that focusing on desired 
outcomes rather than outputs was felt to be beneficial: 

/ think that philosophy really does help because I think 
sometimes you get caught up In looking at what's going in 
. . . rather than whether it is making a difference. 

(Contracts and Review Officer) 

The Turning the O/Ai/e workshops were viewed as 
supportive ways of improving performance because the 
workshops focus on desired outcomes and the stories 
behind current performance. Two interviewees noted 
that Turning the Curve workshops were used because 
they were considered to be the best model for gaining 



buy-in from partners to work together to improve 
outcomes for children and young people. 

When the team considered how to get buy-in from 
partners and how to improve outcomes, there was a 
consensus around OBA and Turning the Curve as being the 
best modei to go with. 

(Local Strategic Partnership Performance Manager) 

Some interviewees observed that OBA was one 
approach of many and other methodologies or 
frameworks could also be useful, particularly those that 
are geared towards cost effectiveness (such as the 
Balanced Scorecard developed by Kaplan and Norton - 
see www.balancedscorecard.org for further 
information). One commented about OBA: 

/ think it is a useful methodology and we have benefited 
from using it but I wouidn't want to stand up and argue 
the case for it being the methodology. 

(Head of Standards and School Effectiveness) 

OBA was viewed as a helpful common-sense approach 
and a methodical way of focusing on outcomes rather 
than processes. However, there were instances of LAs 
only using particular elements of the OBA approach - 
'cherry-picking' the parts that they wished to use - 
such as the Turning the Curve workshops to provide a 
means for getting the relevant agencies around the 
table for brainstorming and planning with the outcome 
as the focus. As the quotation below illustrates, there 
was recognition that only particular elements of OBA 
were being applied: 

We are doing lots of parts of OBA well but we are not 
sticking well enough to the stages of it for me to say OBA 
has definitely helped us to change direction or ultimately 
achieve better outcomes for young people. 

(Children and Young People's Strategy Officer) 

This interviewee also mentioned the issue of identifying 
the impact of using OBA which is discussed in more 
detail in Section 4. 
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3.2 Who was involved? 

The typical model comprised a team of senior 
managers within Children's Services (Tiers 2 or 3) 
leading on OBA implementation with one or two key 
officers to 'champion' the approach more widely and 
involve partners. High-level support from the Children's 
Trust executive board or the Director of Children's 
Services was often available and supported successful 
implementation. This senior commitment was shown 
through, for example, attendance at Turning the Curve 
workshops, supporting the use of OBA as a framework 
for commissioning and verbally raising awareness of 
the approach. In all case-study areas, there were staff 
who had undertaken training in OBA with external 
facilitators. 

The Turning the Ct/A/e workshops, a key part of OBA, 
usually involved a wider range of agencies and 
partners, with practitioners and strategic officers 
working alongside each other. The other agencies 
involved varied depending on the focus, for example, 
where the focus was on reducing levels of teenage 
conceptions there was usually involvement from the 
local Primary Care Trust (PCT) and voluntary sector 
support groups. In some cases, young people and their 
families participated in the Turning the Curve 
workshops. 

3.3 What was OBA being used 
for? 

OBA was being used in the case-study LAs in a variety 
of different ways and to benefit different groups of 
children and young people and different service areas. 
Some Children's Services authorities had applied OBA 
to service areas that were not performing well or 
where they were finding it difficult to make 
improvements and 'turn curves'. OBA was used to get 
the trend (for a particular target or performance area) 
moving in a positive direction. As one interviewee 
commented, they were using OBA 'to address our more 
stubborn targets'. In these cases, OBA was being used 
to help achieve specific Local Area Agreement (LAA) or 
National Indicator (Nl) targets (such as to reduce 



teenage conception rates or to improve achievement at 
the end of the Early Years Foundation Stage). Others 
were applying OBA more broadly, by incorporating it 
into the commissioning process and the monitoring of 
contracts or into the review and development process 
for their CYPP. One LA referred to OBA being applied 
to the Local Safeguarding Children's Board (LSCB) 
business planning process. The LSCB were starting to 
implement report cards to monitor and challenge 
progress. OBA was also being used to improve the 
process of gaining feedback from service users. 

Tables 3.1 and 3.2 show a summary of the areas 
where OBA was identified as being applied within the 
nine Children's Services authorities which participated 
in this research. These have been categorised by the 
research team into population and performance 
accountability where this was considered possible. It is 
worth noting that the interviewees did not make these 
distinctions when describing how they were using OBA 
and therefore it was not always clear which type of 
accountability the examples referred to. For 
clarification: 

• population accountability is about improving 
outcomes for a particular population within a defined 
geographical area 

• performance accountability is about the performance 
of a service and improving outcomes for a defined 
group of service users. 



Table 3.1 Population accountability examples 



Case study 


OBA usage 


1 


To help reduce teenage conception and 
childhood obesity 


3 


To report progress on specific outcomes 
identified as priorities in the CYPP 


3 


To improve educational attainment for a local 
population 


4 


At locality level to support a staying safe project 
and an educational achievement project 


6 


To reduce teenage conception rates 


8 


To reduce teenage conception rates 


9 


To reduce teenage conception rates in a 
particular locality 
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Table 3.2 Performance accountability examples 



Case study 


OBA usage 


1 


To develop and review their CYPP 


1 


To develop and monitor the LSCB annual plan 
through the use of report cards 


1 


To help schools evaluate the Impact of their 
extended services 


1 


For commissioning and decommissioning specific 
services 


2 


For purchasing specific services - particularly 
the performance monitoring of all contracts 


2 


To help thinking around performance indicators 
in a range of service areas including mental 
health services, teenage conception, children's 
centres and occupational therapy 


5 


To monitor commissioned projects and services 
purchased from external agencies 


6 


To support performance monitoring of services 
using report cards 


7 


To develop the LSCB business plan 


9 


To increase school attendance in a particular 
locality 



Box 3.1 provides more detail on using OBA to develop 
and review the CYPP 



Box 3.1 Using OBA to develop the CYPP 

One Children's Trust had built the OBA approach 
into the review and development of their CYPP 
for 2009-1 1 . The development of the CYPP was 
based on a process of needs analysis, 
identification of priorities, consultation and 
review. The OBA approach was used in the middle 
stage - the identification of priorities. Nine 
Turning the Curve workshops were held, involving 
a wide range of partner agencies, and the results 
of these workshops helped to inform the draft 
CYPP. The topics for the workshops were 
determined following consideration of the initial 
needs analysis. The workshops were well attended 
by colleagues from all Council directorates and 
partner agencies including schools, the health 
sector, police and voluntary sector. Each workshop 
identified the outcomes that those attending 
wanted to achieve and the indicators that could 
measure this, and shared background knowledge 
about the issue before brainstorming ideas for 
actions to contribute to achieving the outcomes. 



3.4 How long had OBA been 
used for? 

The use of OBA in the case-study LAs was relatively 
new in some cases - with OBA being used for four or 
five months - but some had more experience and had 
been introduced to the approach two or three years 
ago. Only two of the LAs referred specifically to delays 
in implementing OBA: one mentioned that gaining 
sign-up and gaining full understanding of the OBA 
approach had taken longer than anticipated; another 
mentioned technical issues in ensuring their 
performance management systems could incorporate 
the OBA model. 

There was evidence that the longer OBA was used, the 
more embedded the approach became in LA Children's 
Services practices, such as there being reference to 
OBA in service-level action plans with named 
individuals or groups being responsible for 
championing OBA across the LA or Children's Trust or 
identifying where the approach might be helpful in 
improving outcomes. Additionally, the more OBA was 
used, the less it was labelled as OBA and the more it 
was said to have become incorporated into day-to-day 
practice for identifying outcomes and monitoring 
performance. Lor example, one interviewee referred to 
another example of where the OBA approach was 
being used and commented 'this all lacks the label of 
OBA but is very much driven by that process'. 

Newer users of OBA tended to be applying the 
approach to specific projects or to improve specific 
population outcomes in isolation (such as to reduce 
under- 18 conception rates or to reduce absenteeism in 
schools). Where OBA had only been applied to a 
specific area, there were usually plans to apply it more 
widely in the future. (See Section 5.3 for further details.) 

3.5 How was OBA introduced? 

Interviewees gave an account of how OBA was 
introduced in their service area or Children's Trust and 
details of the individual approaches and circumstances 
are described in the individual case studies (see the 
separate case-study report for further details). Below 
are some of the common scenarios and approaches. 
The impact of these is discussed in Section 4. 
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• Those who would be using OBA attended externally- 
run training on the broad concept of OBA. This could 
involve officers attending an OBA introductory event 
and/or a train-the-trainers event. LG Improvement 
and Development-led information seminars, or the 
individual LA bringing an OBA expert in to 
specifically run training. 

• Officers who planned to use OBA built a business 
case to show the value of using the OBA approach, 
to present to senior management in order to gain 
commitment and to agree on how best to implement 
and use OBA. 

• OBA was introduced to those with responsibility for 
improving outcomes for children and young people 
across the Every Child Matters (ECM) areas, or those 
who were responsible for the wider implementation 
of OBA, such as heads of service. 



• OBA was introduced and promoted to wider groups 
across Children's Services (depending on the planned 
scope for implementation). This was usually through 
workshops and events using the Turning the Curve 
methodology for all partners involved in that service 
area. This is illustrated by the following comment: 

'We just introduced the technigue and encouraged 
people to think about it in their areas.' The purpose 
of the workshops was to familiarise people with the 
OBA conceptual framework and to apply the 
approach to specific priority areas. 

• Action plans and strategies were developed by the 
teams or working groups, based on the outcomes 
from the Turning the Ctyci/e workshops. 

• OBA was used in developing and monitoring 
contracts and service-level agreements. 
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4 What was the impact of using OBA? 



Measuring the impact of using OBA is, as with any measure 
in non-experimentai conditions, probiematic because no 
singie factor is iikeiy to have ied soieiy to any change 
observed, in generai, interviewees were reiuctantto 
attribute any improvement in outcomes for chiidren and 
young peopie specificaiiy to OBA, because OBA was just 
one part of an approach being used to identify and improve 
outcomes. This is iiiustrated in thefoiiowing comment: 

/ think saying OBA leads to an improvement in outcomes is 
realiy misleading; rather, I think it ieads to an improvement 
in the pianning processes to improve outcomes which 
might then lead to improved outcomes because the 
planning is better. It's the actions that you identify and the 
way you drive them that makes a difference rather than 
the actual methodology. 

(Head of Standards and School Effectiveness) 

4.1 What outcomes were 
evidenced as a result of 
using OBA? 

The research team examined the impact data from the 
nine case-study areas by attributing the OBA definitions 



of population and performance accountability (see 
Section 1 for definitions of population and performance 
accountability). 

If OBA had an impact at population level, then 
outcomes will improve for particular populations. If 
OBA had an impact at service level (performance 
accountability level), then service improvements might 
happen in terms of: 

• a change in the number of service users the service 
was working with and/or 

• a change in the quality of the service for the service 
users and/or most importantly 

• a change in outcomes for the service users. 

The research team found limited conclusive evidence of 
improvement in outcomes or service performance from 
the case-study interviews and documentation supplied 
as shown in Tables 4.1 and 4.2. 



Table 4.1 Population accountability 

Case Service(s) Impact Comments 

study 



1 



Early years An overall improvement in 
achievement at the end of 
the Early Years Eoundation 
Stage (EYES) by 5 per cent 
(from 44 to 49 per cent), also 
with the most disadvantaged 
closing the gap by 1.3 per 
cent. 



Achievement is defined by the national 
Target E (NI72) - the percentage of young 
children who achieve a total of at least 78 
points across the Early Years Eoundation 
Stage Profile, and Target H (NI92) - to 
reduce inequality by improving the average 
score of the lowest 20 per cent of results. 



8 Teenage 

conceptions 



A reduction in the number of The interviewee was reluctant to attribute 
teenage conceptions. this reduction specifically to OBA but 

following the use of OBA within this area, 
improvements were evident. 
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Table 4.2 Performance accountability 



Case 

study 



Service(s) 



Impact 



Comments 



5 Locality 


Increases in the percentages 


OBA led to more targeted service 


commissioning 


of children and young people 


provision (e.g. emotional health and well- 


to support 


(from vulnerable groups) who 


being support in schools) leading to a 


vulnerable 


report positive outcomes: e.g. 


reduction in numbers of children and 


groups 


56 per cent have accessed 


young people supported. Support now 




inclusive/diversionary 


offered is focused at an early 




activities; 34 per cent 


intervention/ preventative level on an 




reported improved self- 


ongoing basis rather than large numbers 




esteem/confidence; 59 per 


of children attending short-term activities. 




cent reported now having 


This has led to an increase in the 




someone to talk to; and 22 


numbers of children and young people 




per cent showed improved 


from the traditionally vulnerable groups 




behaviour at home and/or 
school (see Case Study 5 for 
corresponding numbers). 


accessing the support offered. 


9 School 


Increased school attendance 


Again, the interviewee was reluctant to 


attendance 


rates and more positive 


attribute this specifically to OBA but 




attitudes to school 


acknowledged that this was the approach 




attendance. 


used. 


There were also examples where the OBA their working practices. In all nine LAs there was 

approach had been used but there was no evidence of organisational impact. The general 

evidence, at the time of the interviews, of consensus from interviewees was that the changes in 

improved outcomes. One LA officer explained that working practices would, over time, lead to improved 



they had used an OBA approach with a Turning 
the Curve workshop to try and reduce the number 
of teenage conceptions but teenage conceptions 
had not yet decreased. The LA officer noted that 
this did not suggest that OBA was ineffective: 

'We always knew that was going to be hard but 
it doesn't mean the method is flawed in terms of 
using OBA.' Teenage conception was viewed as a 
stubborn issue that required further focus before 
the results would show improvement. 



outcomes for children and young people. 

The research team categorised the organisational 
impact of OBA, evidenced in the case studies, into four 
areas: 

• enhancing communication between 
agencies/partners 

• providing a framework for monitoring performance 



4.2 What was the organisational 
impact? 

Although the evidence of a change in outcomes in the 
areas that had adopted OBA was limited at this stage, 
it was evident that using the OBA approach had 
impacted on working practices in the case-study areas. 
Across the nine Children's Services authorities, the 
interviewees were positive about the impact of OBA on 



• increasing the extent to which service users' views 
were gathered and recorded to inform the 
development of services 

• encouraging a more outcomes-focused approach. 

Examples are given under each of these headings 
below, with an indication of the service area or issue 
being referred to first. 
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4.2.1 Enhancing communication 
between agencies/partners 

• Teenage conception and obesity: OBA provided a 
mechanism for bringing together people from 
different agencies to talk about the same issue from 
their particular angle, to develop a mutual 
understanding of the issue. One interviewee 
commented: 'It gave us the opportunity to share all 
of our information and all of our knowledge about 
an issue.' 

• Commissioning: Improved partnership working with 
greater interaction with partners when developing 
contracts/service-level agreements and better 
accountability. 

• To develop the CYPP: OBA helped to identify areas 
that needed multi-agency intervention that they were 
previously trying to address as single agencies. 

• First-time entrants to the criminal justice system: 

OBA helped to involve partners in the performance 
management and led to improved dialogue. An OBA 
approach was used to examine why there had been 
a reduction of 30 per cent in first-time entrants to 
the criminal justice system (from 350 to 245 young 
people). 

• Safer and healthier home life project: OBA has 
helped the Children's Trust to develop ideas and a 
subsequent action plan for partnership working in a 
population with multiple needs. 

• Teenage conception: OBA has supported 
relationship-building with partners from the PCT and 
building a Children's Trust partnership. 

• Areas of the LAA: Improved partnership working 
between the LA, the PCT and voluntary and 
community sector organisations. 

4.2.2 Providing a framework for 
monitoring performance 

• Local Safeguarding Children's Board: Staff developed 
an OBA-based report card which is presented 
regularly to the LSCB and members use it to monitor 



progress, challenge progress and monitor 
performance. 

• Commissioning: Use of OBA resulted in changes in 
working practices by 'sharpening up' the 
commissioning process and decommissioning 
services that were not performing effectively. One 
interviewee commented: 'That's the key impact that 
OBA has had; people are asking "Does it make any 
difference?" and also "Will you notice if it isn't 
there?"' 

• To monitor the CYPP: Staff used OBA to monitor the 
indicators. 

• To monitor the CYPP: OBA has helped to improve the 
quality of their action planning and helped them to 
develop more effective performance monitoring: 'We 
are monitoring more regularly and we have clearer 
indicators to monitor against.' 

4.2.3 Increasing the extent to which 
service users' views were gathered 
and recorded to inform the 
development of services 

• Mental health services: Use of OBA resulted in 
changed working practices in terms of the type of 
data that is recorded and monitored. Previously data 
was collected on the number of referrals and the 
waiting times, but now they collect data on service 
user views using before- and after-questionnaires to 
measure impact. 

• Commissioning: Use of OBA resulted in more 
structured processes with feedback from children and 
young people becoming an established part of the 
process, which it was not previously. Commissioning 
documents are now more concise and easier to 
understand. 



4.2.4 Encouraging a more outcomes- 
focused approach 

• To develop the CYPP: OBA has helped staff to 
change the way they work to focus on what they are 
trying to achieve rather than the process of delivery. 
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• To develop the CYPP: Staff used OBA to help ensure 
their CYPP was outcomes-based. 

• Educational attainment: One Local Area Partnership 
used OBA to address education issues by narrowing 
the gap. They identified children reading as an 
outcome. This led to local initiatives such as books 
being given away by an ice cream van in specifically 
targeted deprived areas, to encourage reading and 
promote services. 



• Performance management: OBA has enabled the 
Children's Services authority to have a better 
understanding of outcomes and has streamlined the 
various action plans through a report card showing 
accountability and performance. One interviewee 
commented: 'I think the biggest difference is that 
things are probably more proactive.' 

• Teenage conception: OBA has helped the service to 
move towards a more outcomes-focused approach. 
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5 What were the barriers and challenges? 



5.1 Barriers and challenges 

As when any new approach is implemented, there are 
a number of barriers and challenges. The main 
challenges identified were: 

• the need for training and support 

• the time needed to build up confidence and 
experience to use OBA 

• where managers felt threatened by OBA 

• where there was a lack of senior-level support. 

One of the main challenges focused around the need 

for training and support in OBA: 

Some services have found it simple to use, but some services 
found it very difficult to use, because their comprehension 
of measurement was a unit of activity rather than outcomes. 

(Performance Manager) 

The most common area where interviewees felt they 
could have made improvements to increase the impact 
of the OBA approach was through OBA training 
reaching a wider group of professionals. Resource and 
capacity were often cited as reasons for not being able 
to do this. Some interviewees felt that this resulted in 
pockets of OBA expertise and pockets of OBA use, 
rather than a more systematic programme of OBA 
implementation, supported at Chief Executive level and 
built into forward planning across Children's Services. 
The following comment is an example of this: 

There are still some people who are saying that they are 
not quite sure what OBA means, and what It means to 
themselves. So we're having to do quite a lot of back- 
tracking and go over old ground really. So perhaps we 
could have. In the beginning stages, been much . . . clearer, 
and made sure that we got more professionals Involved 
right at the very beginning so that we're all singing from 
the same hymn sheet, rather than It being a cascade effect 
(Quality Compliance Officer) 



Building up the confidence and experience to 

use OBA and to train others in using OBA was time 
consuming and adequate resources and capacity were 
needed to allow for this within individuals' workloads. 

In one case, the LA had appointed an officer to deal 
specifically with championing OBA and implementing 
the approach more widely. This was a way of aiming to 
improve the impact by ensuring OBA remained high on 
the agenda and colleagues remained motivated to use 
and embed the approach. In another case, the LA was 
keen to ensure that OBA work was not viewed as 
additional work, but merely a different way of looking 
at the same issues: 

/ think it's about trying to get away from the mindset that 
we're not asking them to do more work; we're just asking 
them to evidence what they're actually achieving. 

(Quality Compliance Qfficer) 

Lor LAs that were using OBA for contract monitoring 
there was an acknowledgement that some managers 
felt threatened by OBA because the results of it (if it 
showed the service was not leading to the desired 
outcome) could lead to their service being 
decommissioned. But involving such services in Turning 
the Curve workshops was suggested as a supportive 
way of improving performance. 

Senior-level support was important for driving 
forward the approach: 

If we could get good commitment at a higher level I think 
we could do really great stuff ... I feel a bit like a disciple 
sometimes, I feel sometimes I have to be quiet about It 
because It's as If I'm banging on about It again. 

(Children's Trust Development Officer) 

The support of senior management was seen by staff at 
tiers 3 and 4 as an important tool in minimising the 
risk of OBA being perceived as 'just another initiative' 
and therefore the possibility that it would not become 
an embedded part of staff's practice. 
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5.2 How can the barriers and 
challenges be overcome? 

Interviewees made a variety of suggestions for 
overcoming barriers and challenges when using the 
OBA approach. These can be categorised in three 
areas: considerations when planning OBA 
implementation; short-term considerations whilst 
implementing OBA; and long-term considerations. 

5.2.1 Implementing OBA: planning 
considerations 

When planning for implementing OBA, LAs suggested 
the following general principles be followed. 

• Gain commitment at high strategic level and engage 
with key partners, for example through 
demonstrating how the model can help to improve 
performance and outcomes - as one interviewee 
said: 'there's a large amount of scepticism so it's 
about demonstrating real examples' - and by using 
the Turning the Curve workshops as an opportunity 
to share knowledge and understanding and 
demonstrate how the OBA approach can assist. 

• Ensure those leading the implementation are trained 
and fully understand the approach so they are able 
to support others, considering appropriate sources of 
expertise such as LG Improvement and Development, 
Regional Improvement and Efficiency Partnership 
(RIEP) or other authorities. 

• Ensure there is clarity of the aims and reasons for 
implementing OBA and whether it is being used to 
focus on population outcomes or specific services 
'Don't lose sight of the outcome that you want to 
achieve'. 



5.2.2 Implementing OBA - short-term 
considerations 

When planning for implementing OBA, LAs suggested 
the following short-term principles be followed. 

• Involve as many partners as possible in Turning the 
Curve workshops to support shared ownership of the 
approach: 'It might be a bit time consuming but it 
will result in much smoother working.' 



• Ensure there is a common understanding of the OBA 
language: for example, what an outcome is; the 
difference between 'effort' and 'effect'. 

• Ensure those leading the Turning the Curve 
workshops are fully prepared and aware of what 
data is available. 

• Keep practising using the outcomes grids^ for 
performance accountability - they are dynamic 
documents which can be improved. 

• Dedicate time for relevant personnel who are going 
to be using OBA to engage with it and 'champion' it 
across the Children's Trust. 

• Begin by introducing OBA in one or two specific 
service areas or focus on particular population 
outcomes. 

5.2.3 Implementing OBA - long-term 
considerations 

When planning for implementing OBA, LAs suggested 
the following long-term principles be followed. 

• Build OBA into existing systems/strategies to ensure 
it becomes embedded. 

• Embed OBA in performance management by using 
the OBA outcomes grid' to focus monitoring on 
outcomes and how these will be measured. 

• Build OBA into a key officer's role within the LA or 
Children's Trust - someone who can act as a 
champion and maintain the wider commitment to 
the approach. 

• Use examples of success of using OBA to help gain 
more universal commitment. 

Generally there was consensus over the main 
considerations when implementing OBA, but there was 
one area where there were different views. There were 
conflicting views as to whether OBA needed to be 
applied in full, as the core planning tool, or whether it 
could be viewed as a model that could be adapted and 
tailored, using the parts that were most relevant and 
useful. The evidence suggested that the former 
approach supported officers in embedding OBA in LA 
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practice, but the latter more flexible approach was, 
perhaps, easier for smaller teams to use when first 
piloting OBA. 

5.3 What were the next steps? 

All case-study LAs indicated that they planned to 
continue to use OBA. Their next steps were based 
around continuing to further promote, roll-out and 
embed the principles of OBA across Children's Services 
and the LA as a whole, and evaluating the impact of 
OBA locally and making decisions based on the 
evaluation findings. 

In one LA, where OBA had been piloted in one area for 
locality-based commissioning, their next steps involved a 
roll-out across the localities. There was clear 
acknowledgement that this reguired a robust and 
specific training package to be implemented. The training 
was viewed as the first in a number of steps designed to 
embed OBA within Children's Services with a view to: 

• commissioning projects using OBA to drive 
performance and improve the outcomes for the 
children and young people they support 

• achieving locality action plans which focus on 
outcomes and are measured using OBA 

• using OBA more widely across all Children's Services 
in the LA. 



One interviewee anticipated that OBA could be used to 
support 'tough decision-making' such as decisions on 
which of their existing priorities to focus on. This was 
thought to be reguired if there are budget cuts: 

There will be all sorts of decisions to be made about 
current services and how to reshape things, and I think the 
next step could be OBA being used to help bring 
stakeholders together to own that process. 

(Principal Officer for Extended Services and 
Children's Centres) 

Another interviewee observed that OBA would be one 
tool which could support them in identifying strategies 
to address the challenges in their Trust: 

OBA Is not something we should be considering on its 
own. We need to consider it as part of a wider problem- 
solving approach . . . thafs what OBA is to us. It‘s a means 
by which we'd look at really difficult Issues and consider 
how we would tackle them. 

(Local Strategic Partnership Director) 

Note 

4 Outcomes grids are part of the OBA methodology 
consisting of four guadrants used to identify the 
quantity and quality of the effort and the effect. 

These revolve around the three key questions: 1. How 
much did we do? 2. How well did we do it? and 3. Is 
anyone better off? to give performance measures. 
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6 Conclusions 



This research study, focusing on the process and impact 
of implementing the OBA approach in Children's 
Services, found evidence to suggest that, at a 
comparatively early stage in its use, OBA was 
contributing to improved outcomes for children and 
young people in a few instances. In the main, however, 
at this stage the use of OBA was having an impact on 
working practices and indirectly on relationships 
between Children's Services and partners in the 
Children's Trust as the Turning the Curve workshops 
provide an opportunity for partners to come together 
with a common focus. Staff who had been using the 
OBA approach believed that, over time, this would lead 
to improved outcomes for children and young people. 

Although the interviewees had yet to identify a clear 
relationship between the use of OBA and improved 
outcomes for children and young people, in general, 
they were looking at making greater use of OBA across 
Children's Services. This was primarily because they had 
found the process of undertaking a Turning the Curve 
workshop a helpful methodology for structuring 
thinking and moving towards becoming more 
outcomes-focused. Some interviewees also found the 
outcome grids and report cards useful, although these 
were less widely used. OBA was often regarded as one 
of a range of management tools available, but one 
which did not, however, deal with cost effectiveness 
(such as Balanced Scorecard). 



As they are adopting the OBA methodology in a 
Children's Services and Children's Trust setting, the 
interviewees were learning and, through their fuller 
understanding, adapting the approach to enable them 
to make best use of it. This included selecting the 
elements of the approach that assisted them in 
identifying outcomes and improving performance. 

The research suggests that the process of 
implementing OBA benefited from the support of 
Directors of Children's Services and equivalent Tier 1 
(director-level) officers from across the Children's Trust 
partner organisations. Where the OBA approach was 
being applied to the review and development of the 
CYPP and to the process of commissioning services and 
contract monitoring there was more likelihood of OBA 
becoming embedded in Children's Services practices. 

The evidence from the interviews suggests that while 
OBA can appear initially to be a simple approach for 
focusing on outcomes, fuller engagement with the 
approach showed that it is complex. Consequently, it is 
evident that in adopting OBA, Children's Services 
authorities needed to allow time to understand fully 
the concepts and terminology and to ensure these are 
mutually understood. In all nine case-study areas 
training had been undertaken in OBA and it appeared 
that this had been valuable in assisting in the 
understanding of the process. 
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Appendix A - A brief summary of OBA 



This Appendix provides more information about the OBA approach, in particular, the background to OBA, 
its concepts and tools. It is taken directly from the LG Improvement and Development website: 
http://www.idea. gov.uk/idk/core/page.do?pageld=8941 122 and is written by Gillian Pugh. 
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Outcomes Based Accountability: a brief summary 

Gillian Pugh 



Background 

The current focus in British social policy on outcomes or results has been informed by 
the work of Mark Friedman in the United States'. Mark and colleagues visited the 
UK in March 2000 to discuss their experience of applying a “results based” or 
“outcomes based” approach to planning services for children, young people and 
families. The publication that applied this thinking to services in the UK^ was 
influential in determining the outcomes framework that underpins Every Child 
Matters and the 2004 Children Act. 

Since that time staff from around 120 local authorities and a number of voluntary 
organisations have attended events on OBA run by the IDEA. An initial evaluation 
has been published^ and a paper describing the approach and including some 20 case 
studies was published by the DCSF earlier in 2008''. OBA has also featured strongly 
during the fieldwork with over 100 local authorities for the Narrowing the Gap 
project^ and is informing the work of the newly established Centre for Excellence and 
Outcomes (C4EO) based at the National Children’s Bureau. 

Outcomes Based Accountability 

OBA is a conceptual approach to planning services and assessing their performance 
that focuses attention on the results - or outcomes - that the services are intended to 
achieve. 

It is also seen as much more than a tool for planning effective services. It can become 
a way of securing strategic and cultural change: moving organisations away from a 
focus on ‘efficiency’ and ‘process’ as the arbiters of value in their services, and 
towards making better outcomes the primary purpose of their organisation and its 
employees. 

Further distinguishing features of the approach are 

• The use of simple and clear language 

• The collection and use of relevant data 

• The involvement of stakeholders, including service users and the wider 
community, in achieving better outcomes 

• The distinction between accountability for performance of services or 
programmes on the one hand, and accountability for outcomes among a 
particular population on the other. 



' Friedman M (2005) Trying Hard is not Good Enough: how to produce measurable improvements for 
customers and communities Trafford Publishing. 

^ Utting, D, Rose W and Pugh G (2001) Better Results for Children and Families: involving communities 
in planning services based on outcomes NCVCCO 

® McAuley C and Cleaver D (2006) Improving Service Delivery - introducing outcomes-based 
accountability IDeA 

^ Uting D, Painter A and Renshaw J (2008) Better Outcomes for Children and Young People - From 
Talk to Action and Turning the Curve Stories DCSF 

^ Local Government Association (2008) Narrowing the Gap: final guidance year 1 LGA/C4EO 
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Tools in the OBA approach 



Central to the OBA approaeh is a proeess deseribed as “turning the eurve” (see Figure 
1 overleaf). An outcome in need of improvement is described, and then 
consideration is given to what is likely to happen over time if nothing changes. 

Plotted as a graph, this projected trend data provides a baseline against which 
subsequent progress can be measured. By instituting effective and timely action to 
achieve better outcomes, service planners can expect to achieve results that move 
away from the projected baseline, thereby “turning the curve”. 




Baselines allow us to define success as turning the curve 
not point-to-point improvement. 

Source: Friedman (200S) 



Turning the Curve processes can be applied to both population accountability and 
service performance. 

The other process central to this approach is the use of performance management 
categories shown below in Figure 2. The aim is to distinguish between quantity 
and quality, and between inputs, outputs and outcomes or results. The bottom 
right hand box - what proportion of service users are better off (quality of effect) - 
is the most important, but the one that is often neglected. 
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Performance measurement categories 



Figure 2 



Quantity Quality 



How much 
did we do? 


How well 
did we do it? 


Is an 
bette 

# 


yone 

roff? 

% 



Population accountability 



The OBA process for population accountability follows the key stages shown as a 
flowchart in Figure 3. 



Figure 3 



Population accountability 




Source: Friedman (2005) 



Population/users: What group or groups of children and families are to be covered, 
and in what geographical area? 



Results: what outcomes or results are wanted for the population identified? 
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Experience: In choosing indicators (see below), how will the wider community as 
well as the identified population experience - see, hear or feel - the intended 
outcomes?. For example, ‘stay safe’ could be experienced in many different ways, 
including more children walking to school, more families using parks at weekend and 
less abuse from binge drinkers near the local off-licence. 

Indicators: For each outcome selected, there needs to be a set of indicators relevant to 
measuring progress. For example. Standard Achievement Test (SAT) scores at age 
1 1, or GCSE results at age 16, or numbers of teenage pregnancies. 

Baselines: A baseline must be established for each indicator used for OBA. As seen in 
Figure 1, this should, ideally, consist of an ‘historic’ part, showing trends based on the 
most recent available data, and a ‘forecast’ of where the trend will go if current 
services and budgets remain the same. 

Story behind the baselines: What lies behind each set of baseline data? For example, 
if children at primary schools in a particular neighbourhood are achieving results 
below the national and local authority average, why is this? What explains teenage 
pregnancy rates that are twice as high in two local authority wards as the whole of the 
rest of the borough? 

Needs assessment: The Turning the Curve process recommends that the choice of 
target outcomes and selection of indicators for measuring progress should precede the 
assessment of related needs in the relevant population. In Friedman’s view, needs 
assessment should be treated as a tool to help understand the story behind the 
baselines and to plan improved services and programmes that will achieve better 
outcomes. 

Partners: Which stakeholders, including community organisations and local people, 
as well as public, voluntary and private agencies, need to come together to achieve 
better outcomes. 

Action needed to succeed (‘what works ’): Guided by the data and the story behind the 
baseline, what changes would help to bring about the intended better outcomes. This 
might include 

• Evidence concerning effective services in other areas - including examples of 
‘what works’ suggested by rigorous research and shared knowledge 
concerning ‘best’ and ‘promising’ practice. 

• ‘No-cost’ or ‘low-cost’ ideas that may not be accredited by research but have 
the potential to be effective based on experience and knowledge of the 
community. 

• Innovative approaches that can stimulate creative thinking, even if the actual 
ideas seem ‘off the wall’. 

• Solutions based on having a fixed, but generous pot of money available to pay 
for them. 
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Criteria and action plan: Four criteria are suggested that can be used to test the 
potential value of potential components in an action plan: 

• Specificity: Is the proposed action specific enough to be implemented? 

• Leverage: How big a contribution will it make to improving outcomes and 
‘turning the curve’? 

• Values: Is the proposed action fair and ethical? 

• Reach: Is it feasible and affordable? 

Conventional SMART criteria can also be used to assess whether solutions are 
realistic and deliverable. 

Budget. ‘No-cost’ and ‘low-cost’ ideas offer an important means of achieving ‘quick 
wins’. But major changes in services for children, young people and families (like 
other age groups) are likely to involve significant sums of public and possibly other 
money. This process has been used strategically by agencies working in partnership - 
including those responsible for local Children and Young People’s Plans - to make a 
compelling case for change. This, in turn, makes it easier for the partners to secure the 
necessary resources from within and beyond their own agencies. 

Performance accountability 

Performance accountability for services follows similar steps already described for 
population accountability, but starts with identifying the service users - see Figure 4. 



Figure 4 




Seven key questions are identified that organisations should routinely ask themselves, 
questions that can be as useful for staff supervision as they can for monthly or 
quarterly monitoring: 

1 . Who are our users? 
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2. How can we measure if our users are better off? 

3. How can we measure if we are delivering services well? 

4. How are we doing on the most important of these measures? 

5. Who are the partners that have a role to play in doing better? 

6. What works to do better, including no-cost and low-cost ideas? 

7. What do we propose to do? 

In conclusion 



The OBA approach can provide a useful framework, or set of questions, to help 
organisations work collaboratively to ensure that they are planning effectively and to 
know whether they are making a difference to the lives of their service users. 
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Recently published reports 



The Local Government Education and Children's Services Research Programme is carried out by 
the NFER. The research projects cover topics and perspectives that are of special interest to local 
authorities. All the reports are published and disseminated by the NFER, with separate executive 
summaries. The summaries, and more information about this series, are available free of charge 
at www.nfer.ac.uk/research/local-government-association/ 



Mfthofity apprxMchvt 
to xboa< 




local authority approaches to the school admissions 
process 

This report gathers the views of local authority admissions officers on 
the strengths and weaknesses of different approaches, as well as the 
issues and challenges they face in this important area. Key findings 
show that, in most areas, the process was being implemented with 
fairness, efficiency and clarity. 

www.nfer.ac.uk/publications/LAW01/ 




Children and young people's views on web 2.0 
technologies 

This research focused on how web 2.0 technologies allow users to 
share, collaborate and interact with one another. The project explored 
the potential of using these tools to collect the views of young people 
and to involve them in their local community. 

www.nfer.ac.uk/publications/LWT01/ 




Local authorities' experiences of improving parental 
confidence in the special educational needs process 

This research focused on LAs with evidence of good practice in 
supporting children with SEN. Partnership working is enhanced where 
SEN teams have a positive ethos and approach towards parents. LAs 
need to ensure that parents have good quality, face-to-face contact 
with SEN professionals at the earliest possible stage in the process. 

www.nfer.ac.uk/publications/LAM01/ 
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SL1 2DQ, tel; +44 (0)1753 637002, fax; +44 (0)1753 637280, email; book.sales@nfer.ac.uk, 
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Is your local authority using outcomes based accountability for plan- 
ning and managing the performance of services? 

If yes, does it lead to an improvement? 

The Outcomes Based Accountability (OBA) approach uses perform- 
ance management categories that distinguish between 'How much 
did we do?', 'How well did we do it?' and 'Is anyone better off?' 

Based on interviews in nine local authorities, this research identifies 
the impact of the OBA approach on local authority practice in chil- 
dren's services. The report covers: 
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• how OBA is being used 

• the impact of an OBA approach 

• the barriers and challenges to using OBA. 

Although it is too early to assess the full impact of OBA, the research 
identifies a number of issues for consideration by children's services 
authorities. For example, it is recommended that local authorities: 

• gain commitment at high strategic level and engage with key partners 

• ensure there is clarity of the aims and reasons for implementing 
OBA and whether it is being used to focus on population outcomes 
or specific services 

• ensure there is a common understanding of the OBA language, for 
example, what an 'outcome' is, and the difference between 'effort' 
and 'effect'. 

This report is important reading for all local authorities using, or con- 
sidering using, an OBA approach. 
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